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SUMMARY 


The  purpose  of  this  report  is  to  present  research  findings  relating  to  the 
possible  advantages  of  various  graphic  information  presentations  and  the 
use  of  deicision  heuristics  as  aids  to  command  and  control  teams.  The 
study  described  in  this  report  utilized  a  complex  dynamic  task  requiring 
extensive  interaction  among  members  of  three-person  teams.  Team  perform¬ 
ance  was  examined  as  a  function  of  four  presentation  schemes  with  or  with¬ 
out  the  provision  of  suggested  performance  strategies  (heuristics). 

Results  indicated  that,  although  presentation  scheme  did  not  affect  per¬ 
formance,  decision  heuristics  significantly  improved  team  performance, 
particularly:  (a)  when  more  time  was  available  to  utilize  the  heuristics, 
and  (b)  when  teams  were  deciding  whether  or  not  to  utilize  resources  for 
uncertain  targets. 
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Section  1 
INTRODUCTION 


BACKGROUND 

Huber  (1984)  predicts  that  group  decision  making,  which  already  pervades 
business,  educational,  and  government  organizations,  will  need  to  be  even 
more  frequent  and  faster  and  will  need  to  account  for  greater  complexity  in 
the  decision  situation.  That  being  the  case  for  decisions  under  normal 
circumstances,  it  is  clear  that  crisis  management  situations  will  arouse 
even  more  significant  demands.  Steven  Fink,  President  of  Lexicon  Communi¬ 
cations  Corporation,  a  management  consulting  firm  specializing  in  crisis 
management,  insists  that,  "Every  crisis  demands  a  crisis  management  team" 
(Fink,  1986). 

Crisis  teams  require  quick,  reliable,  and  clear  information  to  support 
their  decisions  at  times  when  stress  and  time  pressure  can  lead  to  informa¬ 
tion  overload  and  mi scommunication  (Housel,  Sawy,  and  Donovan,  1986). 
Information  in  command  and  control  situations,  potentially  the  most  stress¬ 
ful  of  crisis  management  situations,  "usually  concerns  the  past,  present, 
and  future  location,  identity,  and  certain  other  attributes  of  various 
objects"  to  which  teams  must  respond  quickly  and  accurately  (Wohl  et  al., 
1984).  Performance  demands  are  necessarily  high  in  spite  of  the  dynamic 
and  stressful  environment. 

Computer-based  support  of  group  decision  making  and  the  explicit  discussion 
of  decision  strategies  have  both  been  offered  as  means  to  improve  group 
performance  (Cats-Baril  and  Huber,  1987).  This  study  examines  the  impact 
on  group  performance  of  variations  in  the  graphic  features  of  a  specific 
computer-based  support  system  and  of  the  explicit  discussion  of  an 
experimenter-suggested  decision  strategy.  In  particular,  this  research 
addresses  the  problem  of  facilitating  group  performance  in  a  dynamic  task 
situation  which  involves  uncertainty,  through  the  use  of  graphic  informa¬ 
tion  presentation  and  decision  heuristics. 
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Previous  research  using  the  Team  Resource  Allocation  Problem  (IRAP)  as  an 
experimental  task  has  shown  that  TRAP  team  performance  was  better  at  moder¬ 
ate  rather  than  high  levels  of  time  stress,  that  performance  was  not 
strongly  affected  when  the  TRAP  was  presented  on  one  large  screen  display 
rather  than  three  individual  CRTs  (brown  and  Leupp,  1985),  and  that  per¬ 
formance  was  better  with  a  graphic  rather  than  an  alphanumeric  presentation 
of  the  task  (Wilson  et  al . ,  1987).  The  intent  of  the  present  study  was  to 
explore  the  impact  on  group  performance  of  variations  in  the  graphic  pres¬ 
entation  and  of  introducing  heuristics  as  decision  aids. 

RELEVANT  LITERATURE 

Group  Decision  Making 

Group  decisions  are  different  from  those  of  individuals.  In  spite  of  the 
potential  problems  in  reaching  a  consensus  and  the  opportunity  for  con¬ 
flict,  a  group  tends  to  make  decisions  of  higher  quality  than  the  average 
performance  of  individual  members  of  the  group  (Miner,  1984;  McGrath, 

1984).  Group  decision  making,  however,  also  tends  to  be  inefficient  and 
slow.  Groups  tend  to  proceed  in  bursts  of  activity,  frequently  jumping 
from  one  issue  to  another  while  individuals  seem  able  to  sustain  concen¬ 
tration  on  a  single  issue  (Fisher,  1980). 

Huber  suggests  that  a  group's  actual  decision-making  effectiveness  equals 
its  potential  effectiveness  plus  gains  resulting  from  group  processes  minus 
losses  resulting  from  group  processes  (Huber,  1982).  Groups  gain  because 
they  have  more  resources,  more  sources  of  information  and  new  ideas,  and 
more  perspectives  for  critical  analysis  of  ideas  (Fisher,  1980).  They  gen¬ 
erate  more  alternatives  and  are  better  at  analyzing  the  relative  advantages 
and  disadvantages  of  the  various  alternatives  (Turoff,  1982). 

On  the  other  hand,  group  processes  can  contribute  to  losses  in  decision 
effectiveness.  For  example,  decision  quality  can  suffer  when  individual 
group  members  dominate  the  group  process  beyond  the  merits  of  their  contri¬ 
butions,  when  group  members  mi scommunicate,  or  when  there  are  group  pres¬ 
sures  to  conform  (Huber,  1982),  The  complexity  of  group  communication 
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limits  the  group's  effectiveness,  especially  when  group  productivity 
depends  on  the  coordinated  efforts  of  group  members  (Hackman  and  Morris 
1983)  . 


Group  decision  processes  vary  according  to  the  task.  Intellective  tasks, 
such  as  the  TRAP  used  in  this  study,  have  a  correct  answer.  For  such 
tasks,  the  decision  scheme  that  best  fits  group  performance  has  been 
labelled  "truth,  supported,  wins"  (McGrath  and  Kravitz,  1982).  In  other 
words,  the  group  accepts  a  solution  as  correct  if  at  least  two  members  know 
it  to  be  correct.  A  solution  presented  by  only  one  member,  even  if  it  is 
the  correct  solution,  may  not  be  readily  accepted  unless  the  solution  can 
be  explained  easily  and  is  intuitively  compelling  once  revealed.  Groups 
accept  solutions  to  "eureka"  problems,  then,  on  a  "truth  wins"  basis 
(McGrath  and  Kravitz,  1982), 


Decision  Making  Under  Uncertainty 

The  environment  for  decision  making  in  real  organizations  is  uncertain.  A 
state  of  uncertainty  exists  when  a  decision  maker  has  incomplete  informa¬ 
tion  on  which  to  base  the  decision.  The  dominant  theory  under  which  study 
of  decision  making  under  uncertainty  has  been  conducted  is  the  subjective 
expected  utility  model  (Slovic,  Fischhoff,  and  Lichtenstein,  1977).  It  has 
been  used  as  both  a  normative  and  a  descriptive  model  of  human  risky  choice 
behavior  (Kahneman  and  Tversky,  1979).  According  to  the  model,  "rational" 
manager’s  weight  the  utilities  of  outcomes  by  their  probabilities  and  choose 
among  alternatives  on  the  basis  of  the  highest  expected  utility  (Muhs  and 
Justis,  1981).  Expected  utility  theory,  however,  has  frequently  been 
criticized  as  inadequate  to  describe  human  behavior.  People  are,  for  exam¬ 
ple,  systematically  biased  in  tneir  perception  of  uncertainty  (Grether, 
1978). 


In  addition  to  being  biased  in  their  judgments  of  uncertainty,  humans  are 
often  inconsistent  in  their  choices.  They  frequently  do  not  use  obvious 
optimal  strategies  for  risky  choice  problems  (Little,  1986).  To  account 
for  human  inconsistencies,  Kahneman  and  Tversky  have  proposed  prospect 
theory  as  an  alternative  to  expected  utility  theory.  According  to  prospect 
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theory,  people  go  through  two  phases  In  choosing  among  prospects:  an 
editing  phase  and  then  an  evaluation  phase  (Kahneman  and  Tversky,  1979). 

In  the  editing  phase,  people  simplify  the  statement  of  the  prospects 
through  such  operations  as  coding  (formulating  the  prospect  in  terms  of 
gains  and  losses  from  the  current  position),  rounding  off  probabilities  and 
outcomes,  and  discarding  extremely  unlikely  outcomes.  In  the  evaluation 
phase,  the  value  of  each  outcome  is  weighted,  not  by  the  probabilities  of 
the  outcomes,  but  by  a  decision  weight  that  tends  to  overweight  very  low 
probabilities  and  to  underweight  all  other  probabilities. 

The  combined  effects  of  the  editing  and  evaluation  phases  account  for  many 
of  the  inconsistencies  humans  exhibit  in  risky  choices.  For  example,  pros¬ 
pect  theory  provides  an  explanation  for  why  humans  tend  to  select  certain 
gains  over  merely  probable  gains  with  comparable  or  even  higher  expected 
value  and  to  select  probable  losses  over  certain  losses  (Kahneman  and 
Tversky,  1979). 

Some  early  research  concerning  risky  choice  behavior  in  groups  suggested 
that  groups  tended  to  take  greater  risks  than  the  group  members  would  as 
individuals,  a  finding  labelled  the  "risky  shift"  (Fisher,  1980).  Subse¬ 
quent  studies,  however,  failed  to  support  the  general i zabi 1 ity  of  the  risky 
shift.  Instead,  it  seems  that  groups  may  shift  ("choice  shift"  or  "group 
shift"),  but  not  always  in  the  risky  direction.  Factors  such  as  the  sig¬ 
nificance  of  the  choice  outcomes  apparently  influence  the  direction  and 
magnitude  of  the  shift  (Muhs  and  Justis,  1981). 

The  difficulties  of  making  decisions  under  dynamic,  uncertain  conditions 
call  for  aids  for  the  decision  maker.  Among  the  alternative  decision  aids 
are  computer-based  graphics  and  decision  heuristics,  which  will  be 
addressed  in  the  following  two  subsections. 

Graphic  Representation 

Since  a  graphic  representation  has  been  established  as  superior  to  a 
tabular  representation  for  presenting  information  in  a  dynamic  group 
decision  task  (TRAP)  (Wilson  et  al.,  1987),  comparisons  among  alternative 
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graphic  representations  are  appropriate.  Graphic  representations  of 
information  can  vary  along  a  variety  of  dimensions.  For  example,  Ives 
(198?)  identified  five  basic  visual  input  channels  available  to  the  human 
information  processing  system:  color,  relative  position,  brightness, 
movement,  and  shape.  Appropriate  use  of  these  channels  has  been  shov^n  to 
aid  in  target  identification,  a  fundamental  aspect  of  tne  TRAP. 

Treisman  and  Gelade's  Feature  Integration  Theory  suggests  that  for  tasks 
such  as  visual  search  and  target  identification  and  localization,  visual 
cues  that  are  conjunctions  of  features  (e.g.,  color  and  shape)  require 
focused  attention  and  seem  to  be  processed  serially.  Cues  having  only  a 
single  feature  (e.g.,  color  alone)  may  be  processed  in  parallel  (i.e.,  more 
quickly).  Further,  when  multiple  conjunctively  coded  objects  are  presented 
under  time  constraints  or  when  attention  may  be  diverted,  an  individual  may 
incorrectly  combine  features  of  unattended  objects  and  falsely  report  the 
presence  of  specified  objects  (illusory  conjunctions).  Thus,  especially 
under  conditions  when  attention  may  be  diverted  or  overloaded,  subjects 
perform  visual  search  tasks  more  quickly  and  accurately  when  the  objects 
vary  on  a  single  dimension  rather  than  as  conjunctions  of  features 
(Treisman  and  Gelade,  1980;  Treismari,  1982).  Because  other  researchers  had 
found  qualitative  changes  in  performance  on  visual  tasks  with  extended 
practice,  Treisman  and  Gelade  examined  the  effects  of  practice  on  searching 
for  conjunctively  coded  objects.  They  found  no  indication  of  movement  from 
serial  to  parallel  processing  over  13  blocks  of  practice  (Treisman  and 
Gelade,  1980). 

Considering  only  single-featured  visual  codes,  both  alphanumerics  and  color 
have  been  evaluated  as  excellent  for  use  in  locating  objects,  although 
alphanumerics  are  superior  for  precise  identification  of  the  object  (Davis 
and  Swezey,  1983).  Christ  (1975)  reviev/ed  42  studies  published  between 
1952  and  1973  on  the  effects  of  color  on  visual  search  and  identification 
performance.  In  ten  studies  on  the  accuracy  of  identifying  objects  in  a 
unidimensional  display,  letters  were  superior  to  color,  and  the  advantage 
for  letters  increased  with  increases  in  the  density  of  the  display  and  with 
decreases  in  exposure  time.  On  the  other  hand,  in  four  studies  on  the  time 
required  to  locate  objects,  color  was  superior  to  letters,  although  the 
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difference  was  less  than  that  found  between  color  and  other  visual  codes 
(size,  brightness,  geometric  shapes,  etc.)- 

In  a  more  recent  set  of  experiments,  Christ  and  Corso  (1983)  determined 
that  there  are  "no  ^lear  and  consistent  advantages  for  any  one  visual  code 
set  over  the  others"  (p.  83).  Where  they  found  differences  in  the  relative 
effectiveness  of  the  codes,  these  differences  depended  on  the  other  display 
conditions  (e.g.,  density  of  the  display),  the  task,  and  the  dependent 
measure  used.  Differences  did  exist  under  those  conditions  most  closely 
related  to  the  conditions  established  for  this  study.  For  locating  a 
specified  object  among  12  randomly  located  objects  in  a  display,  location 
times  were  shorter  for  colored  dots  than  for  letters.  Christ  has  also 
demonstrated  that  extended  practice  with  a  visual  task  tends  to  attenuate 
any  differences  in  performance  based  on  use  of  different  single-featured 
codes  (letters,  digits,  familiar  geometric  shapes,  and  colored  dots) 
(Christ,  1975;  Christ  and  Corso,  1983). 

Heuristics  for  Decision  Making 

Given  that  the  strategies  that  decision  makers  adopt  are  sometimes  biased, 
some  effort  has  been  given  to  examining  the  impact  of  attempts  to  reduce 
the  biases.  Procedures  for  reducing  human  bias  are  collectively  known  as 
debiasing  methods  (Fishchoff,  1982).  The  nature  of  a  particular  debiasing 
method  depends  on  the  perceived  source  of  the  bias:  if  the  bias  results 
from  a  faulty  task,  fix  the  task;  if  the  bias  results  from  a  biased  but 
perfectible  decision  maker,  provide  training/feedback  (Fischoft,  1982). 
Assuming  a  perfectible  decision  maker  in  a  probabilistic  task,  training 
could  take  the  form  of  providing  prescriptive  decision  rules,  or  heuris¬ 
tics,  with  the  intent  of  determining  their  impact  on  decision  performance. 

Cats-Baril  and  Huber  (1987)  examined  the  impact  of  providing  decision-aid 
heuristics  on  the  performance  of  an  i 1 1 -structured  career  planning  task. 

The  task  required  participants  to  identify  career  objectives,  generate 
alternative  strategies  for  achieving  the  objectives,  and  prioritize  the 
alternatives.  Some  participants  were  provided  with  a  purpose-expansion 
heuristic  designed  to  increase  the  number  of  objectives  and  alternatives 
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considered  and  to  explore  the  alternatives  more  thoroughly.  Participants 
with  heuristics  performed  better  both  on  objective  measures  of  the  number 
of  issues  addressed  and  on  expert  evaluations  of  the  quality  of  the  plans 
produced.  The  researchers  called  for  additional  studies  on  the  impact  of 
different  types  of  heuristics  on  tasks  with  different  levels  of  structure 
(Cats-Baril  and  Huber,  1987). 

Johnson  and  Payne  (1985)  have  defined  risky  choice  heuristics  as  rules  that 
systematical ly  simplify  the  choice  among  alternatives  by  disregarding  some 
elements  of  the  problem  space  (ignoring  some  alternatives,  selectively 
examining  outcomes,  ignoring  some  event  information,  etc.).  Different 
heuristics  are  based  on  different  simplifications  of  the  choice.  Using 
computer  simulation,  they  compared  six  heuristics  under  a  variety  of  condi¬ 
tions  for  their  accuracy  (conformity  to  a  choice  using  expected  value)  and 
mental  effort  required.  Their  data  suggested  that  "heuristics,  in  at  least 
some  task  environments,  can  approximate  the  accuracy  of  normative  rules 
with  substantial  savings  in  effort"  (Johnson  and  Payne,  1985). 

Arkes,  Dawes,  and  Christensen  (1986)  studied  conditions  under  which  indi¬ 
viduals  would  choose  not  to  use  helpful  but  imperfect  decision  heuristics 
in  a  probabilistic  task.  In  one  experiment,  subjects  were  told  (correctly) 
that  using  the  heuristics  provided  would  result  in  70  percent  accuracy  in 
their  judgments.  Those  who  were  warned  that  deviating  from  the  heuristic 
would  result  in  degraded  performance  and  who  received  no  monetary  incen¬ 
tives  for  performance  did  best,  matching  the  70  percent  accuracy  level. 
Those  who  were  encouraged  to  try  to  improve  on  the  heuristic  or  who 
received  monetary  incentives  for  performance  deviated  from  the  heuristic 
and  judged  less  accurately.  Those  who  received  no  immediate  feedback  on 
the  accuracy  of  their  judgments  also  outperformed  those  who  did  receive 
feedback;  feedback  about  an  incorrect  judgment  tended  to  cause  deviation 
from  the  heuristic  and  degraded  performance  on  the  following  trial.  In  a 
second  experiment,  the  heuristic  provided  resulted  in  75  percent  accu¬ 
racy.  Those  who  had  expertise  in  the  task  context  (or  who  thought  they 
did)  tended  to  use  the  heuristic  less  and  to  perform  worse  than  those  with¬ 
out  expertise. 
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Team  Resource  Allocation  Problem  (TRAP) 

The  experimental  task  used  in  this  study  was  an  adaptation  of  the  TRAP 
(Brown  and  Leupp,  1985).  The  TRAP  was  designed  to  simulate  both  the 

individual  cognitive  processes  and  the  small  group  interaction  processes 

involved  in  making  command  and  control  decisions.  It  was  derived  from  a 
task  used  by  Pattipati,  Kleinman,  and  Ephrath  (1983)  to  study  dynamic 
individual  decision  making. 

As  adapted  for  this  study,  the  TRAP  is  designed  for  three-person  teams  who 

are  seated  side-by-side  and  allowed  to  communicate  freely.  Each  team  mem¬ 

ber  has  a  four-button  control  box  [cursor  UP  and  cursor  DOWN  to  select  a 
particular  target,  and  START  and  RESET  to  assign  or  withdraw  a  resource 
(Brown  and  Leupp,  1985;  Wilson  et  al.,  1987)].  Identical  TRAP  displays  are 
presented  simultaneously  to  each  team  member  via  computer  graphics  work 
stations.  Figure  1  portrays  a  TRAP  display  at  a  discrete  point  in  time  as 
it  might  appear  using  the  alphabetic  coding  scheme. 

As  many  as  11  targets  may  be  portrayed  at  a  time,  each  appearing  on  a  sepa¬ 
rate  row  in  the  TRAP  display.  A  team  earns  points  by  committing  its 
resources  to  a  target  or  targets  for  the  required  period  of  time.  As 
briefed  to  each  team  during  a  training  session,  their  objective  for  the 
TRAP  was 

to  accumulate  as  many  points  as  possible  as  a  team.  This  means  dis¬ 
cussing  alternatives  with  the  other  members  of  the  team  in  order  to 
make  optimum  selection  of  targets.  As  there  will  be  more  targets 
than  the  team  can  possibly  process,  combinations  of  targets  should  be 
selected  which  optimize  team  performance  and  total  point  count. 

Two  types  of  targets  are  represented  in  the  version  of  TRAP  used  in  this 
study:  certain  and  uncertain  targets.  The  payoff  for  committing  resources 
to  certain  targets  is  known  and  guaranteed.  The  payoff  for  committing 
resources  to  uncertain  targets  is  initially  unknown  and,  once  known,  is  not 
guaranteed.  That  is,  a  specified  probability  exists  that  no  points  will  be 
earned  for  committing  resources  to  uncertain  targets. 
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OPPORTUNITY  WINDOW 


WORK 


Table  1  identifies  the  possible  point  values  for  certain  taryets  and  the 
frequency  with  which  each  target  value  appears  in  a  TRAP  trial.  A  TRAP 
trial  presents  a  total  of  44  certain  targets,  distributed  such  that  each 
team  member  has  an  equal  opportunity  to  participate  in  targets  of  a  given 
value  (Brown  and  Leupp,  1985).  A  small  set  of  targets  of  randomly  selected 
values  is  also  presented  at  the  beginning  and  end  of  each  trial;  perform¬ 
ance  on  these  buffer  targets  is  not  analyzed. 

TABLE  1.  POINT  VALUES  FOR  CERTAIN  TARGETS 


Points  Per  Person 

Total  Points 

Frequency 

1-Person  Targets 

1 

1 

6 

3 

3 

12 

5 

5 

6 

2-Person  Targets 

2 

4 

6 

4 

8 

6 

3-Person  Targets 

1 

3 

2 

3 

9 

4 

5 

15 

2 

Some  targets  may  be  handled  by  an  individual  team  member;  handling  other 
targets  may  require  a  specified  set  of  two  or  all  three  team  members  to 
commit  their  resources.  The  black  squares  in  columns  A,  B,  and  C  in  the 
TRAP  display  (see  Figure  1)  indicate  which  team  members  must  commit 
resources  to  a  target  to  earn  its  points.  Each  team  member  controls  a 
separate  cursor  which  can  be  moved  up  or  down  to  any  row  in  the  member's 
column  on  the  display.  The  cursor  (an  asterisk)  is  green  when  the  team 
member  is  available  to  commit  resources  to  a  target.  Once  the  team 
member's  resources  are  committed,  the  cursor  turns  red.  When  all  required 
members  have  committed  resources  to  a  target,  a  horizontal  black  bar 
appears  in  that  target's  row.  The  bar  graphically  indicates  how  much 
longer  resources  must  be  committed  to  the  target  to  earn  its  points.  When 
the  required  time  has  elapsed,  points  for  the  target  are  automatically 
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added  to  the  accumulated  points  table  on  the  display,  and  the  resources  are 
available  for  assignment  to  another  target. 

Four  versions  of  the  TRAP  were  developed  to  reflect  the  four  graphic  pres¬ 
entation  formats  to  be  tested.  Each  version  contains  the  same  mix  of  tar¬ 
gets;  only  the  visual  representation  of  the  certain-valued  targets 
varies.  The  original  TRAP  format  was  retained  as  a  base  for  comparison; 
Table  2  presents  the  types  of  targets  used  in  the  original  TRAP.  Using  the 
points  available  per  person  in  the  original  format  as  coding  levels,  alter¬ 
native  five-level  color,  alphabetic,  and  conjunctive  (color/alphabetic) 
coding  schemes  were  developed  as  shown  in  Table  3.  A  3-person  red  triangle 
in  the  original  version,  then,  appears  as  a  3-person  red  target  in  the 
color  version,  as  a  3-person  A  in  the  alphabetic  version,  and  so  forth. 

TABLE  2.  TRAP  TASK  POINT  VALUES,  ORIGINAL  PRESENTATION  STYLE 


1-Person  Tasks 
(A,  B,  or  C) 


Point  Value 


3 


5 


2-Person  Tasks 
(AB,  AC,  or  BC) 


Point  Value 
(Per  Person) 


3-Person  Tasks 
(ABC) 


Point  Value 
(Per  Person) 


or 


4 

(2) 


8 

(4) 
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TABLE  3. 

ALTERNATIVE 

TRAP  CODING 

SCHEMES 

1 

Points  Per  Person 
2  - 3 - 

4 

5 

Color 

Blue 

Green 

Yellow 

Orange 

Red 

A1 phabetic 

E 

0 

C 

B 

A 

Conjunctive 

B1  ue 

Blue 

B1  ue 

Red 

Red 

C 

B 

A 

B 

A 

Mixed  randomly  among  the  44  certain  targets  in  each  trial  are  20  uncertain 
targets,  represented  as  rectangles  ("black  boxes")  in  all  four  versions  of 
the  TRAP.  Participants  get  additional,  though  imperfect,  information  about 
an  uncertain  target  if  all  three  members  move  their  cursors  to  the  target 
row  and  press  their  start  buttons.  After  a  brief  delay  (one-fifth  the  time 
required  to  complete  a  target),  the  number  of  points  possible  for  com¬ 
pleting  the  target  and  its  probability  of  payoff  appear  on  the  black  box. 

At  that  point,  the  team  may  choose  to  commit  resources  to  that  target  or 
not  as  with  any  other  target. 

The  probability  of  payoff  is  either  80  percent  or  20  percent,  shown  as 
H  (high)  or  L  (low)  on  the  black  box  after  querying  the  system.  The  pos¬ 
sible  values  for  high  probability  targets  are  9  and  21  points,  and  the  pos¬ 
sible  values  for  low  probability  targets  are  36  and  84  points.  Therefore, 
the  expected  value  of  an  H21  or  an  L84  is  16.8  points,  and  the  expected 
value  of  an  H9  or  an  L36  is  7.2  points.  Each  of  the  four  types  of  uncer¬ 
tain  targets  is  equally  likely  to  occur.  Considering  the  time  delay  to 
query  the  system,  an  uncertain  target  is  equivalent  to  a  10-point  certain 
target  for  a  team  that  chooses  to  complete  every  uncertain  target  it  que¬ 
ries.  By  completing  only  the  H21  and  L84  targets  it  identifies,  a  team  can 
make  uncertain  targets  equivalent  in  the  long  run  to  12-point  certain 
targets. 


Based  on  analysis  of  the  performance  of  teams  in  earlier  studies,  inter¬ 
views  with  some  team  members,  and  information  about  the  structure  of  the 
TRAP,  a  set  of  heuristics  was  developed  aimed  at  providing  expert  guidance 


for  performing  the  "'RAP.  Statements  of  the  heuristics  were  tailored  to 
each  of  the  graphic  formats.  Stated  in  terms  appropriate  to  the  original 
version,  the  heuristics  are  as  follows: 

1.  In  addition  to  3-person  red  triangles,  look  for  a  2-person  red 
target,  especially  with  a  1-person  red  circle. 

2.  Next,  check  as  many  uncertain  targets  as  you  can.  Immediately 
take  L84  or  H21;  ignore  L36  and  H9. 

3.  Ignore  blue  targets  except  1-person  blue  circles  and  3-person 
blue  triangles. 

4.  Keep  the  team  synchronized.  All  three  team  members  should  start 
a  target  or  targets  at  the  same  time. 

As  in  earlier  studies,  good  performance  on  the  TRAP  depended  on  analyzing 
the  available  options  and  choosing  those  targets  that  earned  the  most 
points  for  the  team.  Cooperative  thinking  and  action  was  essential;  for 
example,  skipping  a  high  value  1-person  target  to  participate  in  a  2-person 
or  3-person  target  was  sometimes  appropriate. 
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Section  2 
METHOD 


SUBJECTS 

The  subjects  were  96  paid  volunteers,  primarily  students  from  the 
University  of  Dayton  and  Wright  State  University.  All  had  at  least  6/12 
corrected  vision  (6/21  vision  was  adequate  to  perform  the  TRAP).  All  were 
screened  for  color  vision  using  Ishihara  plates;  in  those  few  cases  where 
initial  screening  suggested  color  vision  limitations,  additional  screening 
during  the  training  session  assured  that  all  participants  could  properly 
distinguish  among  features  on  the  display.  No  subjects  who  had  partici¬ 
pated  in  previous  TRAP  studies  were  eligible  to  participate  again.  Sub¬ 
jects  were  assigned  to  teams  based  on  their  availability  to  participate. 
Eight  teams  were  assigned  randomly  to  each  of  the  four  graphic  presentation 
formats.  Among  each  of  the  8-team  groups,  half  were  randomly  selected  to 
receive  the  heuristics. 

EQUIPMENT 

The  displays  were  controlled  by  a  Silicon  Graphics  Model  2400  graphics  dis¬ 
play  processor.  Control  response  buttons  on  the  custom-built  response 
boxes  interfaced  with  the  computer  via  an  analog-to-digital  converter.  The 
three  individual  displays  were  Conrac  Model  7211,  33  cm  diagonal,  full 
color  raster  scan  CRTs  with  resolution  of  921  horizontal  by  739  vertical 
pixels,  and  40  MHz  video  bandwidth.  Character  height  on  the  CRTs  was 
.48  cm,  subtending  an  angle  of  22.9  minutes  of  arc  at  the  46  cm  viewing 
distance. 

PROCEDURE 

A  training  session  of  approximately  2  hours  preceded  experimental  par¬ 
ticipation.  The  first  stage  of  the  training  session  replicated  training 
given  in  earlier  TRAP  studies  (Brown  and  Leupp,  1985;  Wilson  et  al., 

1987).  Participants  were  randomly  assigned  to  seating  positions,  which 
were  maintained  throughout  the  study.  Subjects  were  seated  side  by  side. 
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and  each  subject  had  a  CRT  and  a  response  box.  Participants  wore  headsets 
with  microphones  for  recording  team  communication. 

The  subjects  received  thorough  instructions,  tailored  to  the  presentation 
format  assigned  to  the  team.  The  instructions  described  the  TRAP  and  pre¬ 
sented  the  rules  determining  the  point  values  of  targets.  Before  pro¬ 
ceeding  with  training,  subjects  were  tested  on  their  understanding  of  the 
point  values.  Any  incorrect  responses  were  reviewed,  and  the  subjects  were 
retested  until  they  could  correctly  identify  all  target  values.  Next, 
during  a  slow  speed  demonstration  of  a  TRAP  trial,  further  instruction 
described  features  of  the  display,  taught  subjects  how  to  use  their 
response  boxes,  and  presented  rules  for  working  on  targets.  The  first 
stage  of  training  ended  with  two  practice  trials,  one  under  moderate  time 
pressure  and  one  under  high  time  pressure.  Subjects  were  free  to  ask  ques¬ 
tions  of  the  experimenter  during  the  instruction  and  practice  and  to  com¬ 
municate  with  one  another  during  this  and  all  subsequent  sessions. 

The  second  stage  of  training  introduced  the  uncertain  targets  to  all  teams 
and  the  heuristics  to  half  of  the  teams.  During  a  slow  speed  demonstration 
trial,  instructions  described  the  appearance  of  the  uncertain  targets  and 
provided  the  rules  for  querying  and  committing  resources  to  them.  All 
teams  were  told  about  the  four  possible  types  of  uncertain  targets;  that 
each  type  was  equally  likely  to  occur;  and  that,  considering  the  time  delay 
to  query  the  system,  the  uncertain  targets  were  about  equivalent  to  10- 
point  certain  targets.  They  were  reminded  that  the  objective  remained  to 
maximize  team  performance.  Those  teams  assigned  to  the  heuristics 
treatment  groups  also  received  instruction  on  the  heuristics  appropriate  to 
their  presentation  format.  The  instructions  included  the  rationale  for 
each  of  the  heuristics,  including  an  introduction  to  the  concept  of 
expected  value.  The  teams  with  heuristics  were  tested  to  assure  that  they 
understood  the  implications  of  the  heuristics  on  decisions  in  various  TRAP 
situations.  They  were  also  encouraged  to  discuss  the  heuristics  along  with 
any  other  decision  strategies  they  might  think  to  be  appropriate.  All 
teams  concluded  their  training  sessions  with  four  practice  trials. 
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Hdch  of  the  two  2-hour  experimental  sessions  consisted  of  a  brief  review,  a 
test  session  made  up  of  eight  trials,  a  short  break,  and  a  second  test  ses¬ 
sion.  The  review  involved  testing  (with  review  and  retesting  as  required) 
on  target  values  (all  teams)  and  on  using  the  heuristics  (teams  with  heur¬ 
istics  instruction  only).  Each  trial  in  the  test  sessions  consisted  of  64 
targets  surrounded  by  a  buffer  of  targets  at  the  beginning  and  end  of  the 
trial;  buffer  targets  were  not  analyzed.  In  each  test  session,  there  were 
four  fast  trials  (a  new  target  appeared  every  2.73  seconds  and  remained  on 
the  screen  for  30  seconds,  of  which  10  seconds  of  required  resources  were 
needed  for  completion)  and  four  moderate  speed  trials  (a  new  target 
appeared  every  5.45  seconds  and  remained  on  the  screen  for  60  seconds,  of 
which  20  seconds  of  required  resources  were  needed  for  completion).  The 
sequence  of  fast  and  moderate  speed  trials  was  randomized  in  blocks  of  four 
trials.  The  sequence  of  targets  within  a  trial  was  randomized  for  each 
trial . 

Scoring  for  the  uncertain  events  vanes  from  that  for  other  events.  A  team 
that  commits  resources  to  an  uncertain  event  expects  to  receive  all  or  none 
of  the  possible  points  for  the  event,  depending  on  whether  or  not  the  event 
pays  off.  Therefore,  scores  displayed  to  the  team  reflect  award  of  all  or 
none  of  the  possible  points.  Of  the  five  H9  events  available  in  a  trial, 
for  example,  one  is  randomly  selected  in  advance  to  pay  no  points  should 
the  team  commit  resources  to  that  event. 

The  researcher,  however,  is  interested  in  comparing  the  quality  of  the 
decision  making  rather  than  degrees  of  luck  among  the  teams.  Scores  could 
be  distorted,  for  example,  if  a  team  randomly  received  a  large  number  of 
points  for  committing  resources  to  a  few  low  probability  events  or  if  a 
team  failed  to  gain  any  points  despite  committing  resources  to  a  number  of 
high  probability  events.  Therefore,  separate  scores  are  computed  for 
research  purposes  which  add  the  expected  value  (the  probability  of  payoff 
times  the  possible  point  value)  of  each  completed  uncertain  event  to  the 
accumulated  points  for  the  certain  events.  Teams  should  elect  to  commit 
resources  to  uncertain  events  only  when  the  events'  expected  values  exceed 
the  values  of  other  events  on  the  display.  Therefore,  scores  based  on 
expected  values  more  accurately  represent  decision  making  quality. 


16 


Section  3 
RESULTS 


This  section  presents  the  team  performance  findings,  proceeding  from  an 
analysis  of  an  overall  measure  of  performance  to  a  more  detailed  look  at 
factors  that  contributed  to  overall  performance. 

A  computer  model  that  uses  "correct"  logic  in  performing  the  TRAP  has  been 
developed  as  a  basis  for  comparison  with  performance  by  human  teams.  The 
computer  model  compares  values  of  all  alternatives  and  commits  resources  to 
the  highest  value  event  or  combination  of  events  available  at  a  point  in 
time.  It  repeats  its  assessment  and  resource  commitment  cycle  each  time 
its  resources  are  freed  throughout  a  trial.  For  research  purposes,  overall 
team  scores  are  stated  as  ratios  of  the  points  earned  by  the  team  to  the 
points  amassed  by  the  model.  Team  scores  reflecting  proportions  of  each 
event  type  completed  are  also  compared  to  model  results.  Relating  team 
scores  to  "best  possible"  scores  helps  to  alleviate  any  bias  associated 
with  variations  in  the  difficulty  of  the  experimental  trials,  because  dif¬ 
ficulty  may  vary  due  to  the  randomized  sequencing  of  events.  For  example, 
if  many  targets  of  high  value  occur  together,  neither  actual  teams  nor  the 
model  can  possibly  complete  all  of  them.  Therefore,  team  score  relative  to 
the  score  of  the  model  provides  a  more  accurate  measure  of  the  quality  of 
team  performance. 

The  ratio  of  the  raw  team  score  to  the  score  obtained  by  applying  the 
computer  model  to  the  same  sequence  of  targets,  an  overall  measure  of 
performance,  was  analyzed  as  a  function  of  presentation  style  (original, 
color,  alphabetic,  conjunctive),  heuristics  (with,  without),  time  stress 
(high,  moderate),  and  session  (1,  2,  3,  4).  Next,  a  component  of  the 
score,  the  proportion  of  a  team's  points  earned  on  certain  targets,  was 
analyzed  as  a  function  of  the  same  factors.  Finally,  performance  was 
examined  in  terms  of  the  proportions  of  various  types  of  certain  and 
uncertain  targets  completed. 
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TEAf'l  SCORE 


The  team  score,  expressed  as  a  proportion  of  the  computer  model  score,  was 
influenced  by  heuristics,  time  pressure,  session,  and  the  interaction 
between  heuristics  and  time  pressure.  Teams  with  heuristics  had  higher 
scores  (75.4  percent  of  model  score)  than  teams  without  heuristics 
(70.4  percent  of  model  score),  p  =  .0011.  Teams  also  had  higher  scores 
under  moderate  time  pressure  (79.0  percent  of  model)  than  under  high  time 
pressure  (66.9  percent  of  model),  p  =  .0001.  Scores  improved  from  ses¬ 
sion  1  (70.2  percent  of  model)  to  session  2  (72.3  percent  of  model)  and 
again  to  session  3  (74.3  percent  of  model)  and  then  stabilized  in  session  4 
(74.8  percent  of  model),  p  =  .0001.  Finally,  heuristics  had  a  greater  pos¬ 
itive  influence  under  moderate  time  pressure  than  under  high  time  pressure, 
p  =  .00^0  .  Under  moderate  time  pressure,  teams  with  heuristics  averaged 
82.3  percent  of  the  model's  score  while  teams  without  heuristics  averaged 
75.7  percent,  a  difference  of  6.6  percent.  Under  high  time  pressure,  teams 
with  heuristics  averaged  68.6  percent  of  the  model's  score  while  teams 
without  heuristics  averaged  65.1  percent,  a  difference  of  3.5  percent. 
Contrary  to  expectations,  there  were  no  significant  differences  in  score 
associated  with  the  style  of  graphic  presentation,  p  =  .5669. 

COMPONENT  SCORE 

The  portion  of  the  team  score  achieved  by  completing  certain-valued  targets 
was  influenced  by  heuristics,  session,  the  interaction  between  heuristics 
and  time  pressure,  the  interaction  between  heuristics  and  session,  and  the 
three-way  interaction  among  presentation  style,  heuristics,  and  session. 

For  the  sake  of  comparison,  the  model  consistently  obtained  45  to  46  per¬ 
cent  of  its  points  from  certain  targets.  Teams  with  heuristics  got  a  lower 
proportion  of  their  points  from  certain  targets  (42.0  percent)  than  did 
teams  without  heuristics  (59.3  percent),  p  =  .0071.  The  proportion  of 
points  from  certain  targets  was  lower  in  session  4  (47.4  percent)  than  in 
either  session  1  (52.1  percent)  or  session  2  (53.9  percent),  p  =  .0081, 
with  session  3  in  between  (49.3  percent). 
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Differences  in  time  pressure  had  a  greater  impact  on  teams  without  heuris¬ 
tics  than  on  teams  with  heuristics,  p  =  .0237.  Teams  with  heuristics 
averaged  41.6  percent  of  their  points  from  certain  targets  under  high  time 
pressure  and  42.5  percent  under  moderate  time  pressure.  Teams  without 
heuristics  averaged  61.5  percent  from  certain  targets  under  high  time 
pressure  and  57.1  percent  from  certain  targets  under  moderate  time  pres¬ 
sure.  The  interaction  between  heuristics  and  session  indicates  that  the 
difference  in  proportions  of  scores  from  certain  targets  between  teams  with 
heuristics  and  those  without  was  greater  in  sessions  1  and  2  than  in  ses¬ 
sions  3  and  4,  p  =  .0026  (Figure  2  and  Table  4).  Using  the  averages  in 
Table  4,  the  differences  in  the  percentage  of  points  from  certain  targets 
are  as  follows;  session  1,  22.7  percent;  session  2,  22.8  percent;  ses¬ 
sion  3,  13.2  percent;  session  4,  10.5  percent.  The  three-way  interaction 
among  presentation  style,  heuristics,  and  session  shows  that  the  conver¬ 
gence  in  scores  from  certain  targets  between  teams  with  heuristics  and 
those  without  heuristics  occurred  at  different  rates  for  teams  using  dif¬ 
ferent  presentation  styles,  p  =  .0186.  The  interaction  is  illustrated  in 
Figure  3  and  detailed  in  Table  5.  The  differences  in  Table  5  show  the  fol¬ 
lowing  patterns: 

•  Teams  using  the  alphabetic  style  with  or  without  heuristics  never 
differed  greatly  in  proportion  of  points  from  certain  targets 
across  the  four  sessions. 

0  The  proportion  of  points  from  certain  targets  for  teams  using  the 
conjunctive  style  with  and  without  heuristics  converged  at  a 
smooth  rate  across  the  four  sessions. 

0  The  proportion  of  points  from  certain  targets  for  teams  using  the 
original  style  with  and  without  heuristics  converged  most  quickly 
from  session  2  to  session  3. 

0  The  proportion  of  points  from  certain  targets  for  teams  using  the 
color  style  with  and  without  heuristics  diverged  in  session  2, 
then  started  to  converge  in  sessions  3  and  4  without  ever  closing 
the  gap  as  much  as  teams  using  other  styles  had. 
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12  3  4 


SESSION 


Figure  2.  Proportion  of  Points  from  Certain  Targets  as  a  Function  of 
Heuristics  and  Session 


TABLE  4.  PROPORTION  OF  POINTS  FROM  CERTAIN  TARGETS 
AS  A  FUNCTION  OF  HEURISTICS  AND  SESSION 


Session  1 

Session  2 

Session  3 

Session  4 

With 

40.8  Percent 

42.5  Percent 

42.7  Percent 

42.2  Percent 

Without 

63.5  Percent 

65.3  Percent 

55.9  Percent 

52.7  Percent 
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OBO  H  -  COLOR 


HEURISTICS;  WITH  WITHOUT  WITH  WITHOUT  WITH  WITHOUT 


TABLE  5.  DIFFEKEUCES  IN  PKOPORTIOW  OF  POINTS  FKai  CERTAIN 
TARGETS  BETWEEN  TEAI4S  WITH  AND  WITHOUT  HEURISTICS 
AS  A  FUNCTION  OF  STYLE  AND  SESSION 


Session  1 

Session  2 

Session  3 

Session  4 

Color 

26.0 

Percent 

40.3 

Percent 

33.4 

Percent 

24.4 

Percent 

Alphabetic 

1.7 

Percent 

4.5 

Percent 

0.6 

Percent 

5.5 

Percent 

Conjunctive 

19.7 

Percent 

14.0 

Percent 

8.0 

Percent 

2.6 

Percent 

Ori gi nal 

43.2 

Percent 

32.4 

Percent 

10.5 

Percent 

9.7 

Percent 

PROPORTIONS  OF  KEY  CERTAIN  TARGETS  COMPLETED 

Of  the  eight  types  of  certain  targets  included  in  the  TRAP,  five  were 
specifically  addressed  by  the  heuristics.  Teams  with  heuristics  were 
encouraged  to  work  on  two  types:  3-person-5-poi nts-per-person  targets  and 

2- person-4-poi nts-per-person  targets  (especially  in  combination  with  high- 
value  1-person  targets).  They  were  also  encouraged  to  ignore  three 
types:  1-person-l-poi nt-per-person,  2-person-2-points-per-person,  and 

3- person-l-point-per-person  targets.  Most  teams,  with  or  without  heuris¬ 
tics,  completed  no  3-person-l-point-l-per-person  targets,  so  these  targets 
could  not  be  analyzed  using  ANOVA  procedures.  Of  the  17  such  targets  com¬ 
pleted,  12  were  completed  by  one  team  using  the  original  presentation  style 
without  heuristics.  The  remaining  5  targets  were  spread  over  4  other 
teams:  3  targets  by  2  other  teams  using  the  original  style  without  heuris¬ 
tics,  1  target  by  a  team  using  the  original  style  with  heuristics,  and  1 
target  by  a  team  using  the  conjunctive  style  with  heuristics.  Proportions 
completed  of  the  other  four  types  of  certain  targets  were  analyzed  as  func¬ 
tions  of  presentation  style,  heuristics,  time  stress,  and  session. 

3-Person-5-Poi nts-Per-Person 


The  computer  model  completed  99.95  percent  of  the  15-point  targets.  No 
significant  differences  were  found  in  the  proportion  of  these  highest- 
valued  certain  targets  completed  based  on  any  of  the  factors  analyzed. 
Teams  completed  nearly  all  of  these  targets  under  any  conditions. 
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2-Person-4-Poi nts-Per-Person 


The  computer  model  completed  58.9  percent  of  the  8-point  targets.  Time 
pressure,  the  interaction  between  heuristics  and  time  pressure,  and  the 
interaction  between  heuristics  and  session  influenced  the  proportion  com¬ 
pleted  by  the  teams.  Teams  completed  more  8-point  targets  under  moderate 
time  pressure  (41.9  percent)  than  under  high  pressure  (34.3  percent), 
p  =  .0001.  Under  high  time  pressure,  teams  with  heuristics  completed  fewer 
8-point  targets  (30.0  percent)  than  did  teams  without  heuristics  (38.6  per¬ 
cent),  but,  under  moderate  time  pressure,  teams  with  or  without  heuristics 
completed  about  the  same  number  (with,  41.3  percent;  without,  42.6  per¬ 
cent),  p  =  .0402.  The  interaction  between  heuristics  and  session  indicates 
that  the  difference  in  proportion  of  8-point  targets  completed  between 
teams  with  heuristics  and  those  without  was  greater  in  sessions  1  and 
2  than  in  sessions  3  and  4,  p  =  .0056.  Using  the  averages  in  Table  6,  the 
differences  are  as  follows:  session  1,  12.8  percent;  session  2,  9.2  per¬ 
cent;  session  3,  1.6  percent;  and  session  4,  -3.9  percent. 

TABLE  6.  PROPORTION  OF  8-POINT  TARGETS  COMPLETED 
AS  A  FUNCTION  OF  HEURISTICS  AND  SESSION 


Session  1 

Session  2 

Session  3 

Session  4 

With 

31.1  Percent 

37.3  Percent 

36.0  Percent 

38.2  Percent 

Without 

43.9  Percent 

46.5  Percent 

37.6  Percent 

34.3  Percent 

1-Person-l-Poi nt-Per-Person 


The  computer  model  completed  7.3  percent  of  the  1-point  targets.  Presenta¬ 
tion  style,  heuristics,  and  time  pressure  influenced  the  proportion  com¬ 
pleted  by  the  teams.  Teams  using  the  original  presentation  style  completed 
more  1-point  targets  (15.8  percent)  than  did  teams  using  any  other  style 
(color,  6.6  percent;  alphabetic,  5.4  percent;  conjunctive,  4.8  percent), 
p  =  .0209.  Teams  with  heuristics  completed  fewer  1-point  targets  (4.4  per¬ 
cent)  than  did  teams  without  heuristics  (11.9  percent),  p  =  .0080. 

Finally,  teams  completed  more  1-point  targets  under  moderate  time  pressure 
(9.0  percent)  than  under  high  time  pressure  (7.3  percent),  p  =  .0158. 
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2-Person-2-Poi nts-Per-Person 


The  computer  model  completed  4.2  percent  of  the  4-point  targets.  Heuris¬ 
tics  and  time  pressure  influenced  the  proportion  completed  by  the  teams. 
Teams  with  heuristics  completed  fewer  4-point  targets  (1.6  percent)  than 
did  teams  without  heuristics  (6.3  percent),  p  =  .0142.  Teams  completed 
more  4-point  targets  under  moderate  time  pressure  (4.6  percent)  than  under 
high  time  pressure  (3.4  percent),  p  =  .0115. 

PROPORTIONS  OF  UNCERTAIN  TARGETS  COMPLETED 

Proportions  of  uncertain  targets  completed  were  analyzed  in  two  stages. 
First,  the  proportion  of  all  uncertain  targets  about  which  teams  sought 
additional  information  was  analyzed  as  a  function  of  heuristics,  type  of 
target  (H21,  H9,  L84,  L36),  time  pressure,  and  session.  Second,  the  pro¬ 
portion  of  uncertain  targets  completed  subsequent  to  seeking  additional 
information  was  analyzed  as  a  function  of  the  same  factors.  Because  uncer¬ 
tain  targets  were  presented  as  black  rectangles  in  all  presentation  styles, 
and  because  there  were  no  significant  effects  associated  with  style  on 
querying  or  completing  uncertain  targets,  style  was  not  included  in  the 
reported  analyses  of  uncertain  targets. 

Queries  of  Uncertain  Targets 

The  computer  model  sought  additional  information  about  65.7  percent  of  the 
uncertain  targets.  The  proportion  of  uncertain  targets  queried  by  the 
teams  was  influenced  by  heuristics,  time  pressure,  session,  and  the  inter¬ 
action  between  heuristics  and  session.  Teams  with  heuristics  queried  more 
uncertain  targets  (57.9  percent)  than  did  teams  without  heuristics 
(35.8  percent),  p  =  .0002.  Teams  queried  more  uncertain  targets  under  mod¬ 
erate  time  pressure  (51.0  percent)  than  under  high  time  pressure  (42.6  per¬ 
cent),  p  =  .0001.  Teams  queried  more  uncertain  targets  in  session  4 
(50.2  percent)  than  in  sessions  1  (44.9  percent)  or  2  (44.7  percent), 
p  =  .0036,  with  session  3  in  between  (47.5  percent).  The  interaction 
between  heuristics  and  session  indicates  that,  although  teams  with  heuris¬ 
tics  queried  about  the  same  number  of  uncertain  targets  in  each  session. 
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teams  without  heuristics  queried  more  uncertain  targets  in  sessions  3  and  4 
than  in  sessions  1  and  2  (Figure  4  and  Table  7),  p  =  .0451. 

Uncertain  Targets  Completed 

The  proportion  of  uncertain  targets  completed  was  expressed  as  a  ratio  of 
targets  of  a  particular  type  completed  to  targets  of  that  type  queried. 

Five  teams  failed  to  query  at  least  one  uncertain  target  of  each  type  at 
each  level  of  time  pressure  in  each  session  and,  therefore,  had  at  least 
one  ratio  of  0/0.  After  eliminating  the  five  teams,  a  preliminary  analy¬ 
sis,  using  a  general  linear  model  to  account  for  unequal  cell  sizes,  indi¬ 
cated  that  there  was  no  effect  due  to  session  or  to  any  interactions 
involving  session  (p  >  .05).  Therefore,  data  were  averaged  across  ses¬ 
sion.  Every  team  queried  at  least  one  of  each  type  of  uncertain  target 
during  the  experiment  (no  teams  had  to  be  eliminated),  so  an  ANOVA  model 
including  only  time,  type  of  target,  and  heuristics  as  independent  vari¬ 
ables  could  be  analyzed.  Using  the  new  ANOVA  model,  ratios  of  uncertain 
targets  completed  were  influenced  by  heuristics,  type  of  target,  and  the 
interaction  between  heuristics  and  type  of  target. 

Overall,  the  computer  model  completed  52.3  percent  of  the  uncertain  targets 
it  queried.  Teams  with  heuristics  completed  a  smaller  proportion  of  the 
uncertain  targets  they  queried  (46.8  percent)  than  did  teams  without  heur¬ 
istics  (63.3  percent),  p  =  .0005.  Of  the  various  types  of  uncertain  tar¬ 
gets  it  queried,  the  computer  model  completed  99.97  percent  of  the  H21 
targets,  100  percent  of  the  L84  targets,  5.0  percent  of  the  H9  targets,  and 
4.3  percent  of  the  L36  targets.  Teams  completed  a  larger  proportion  of  the 
H21  and  L84  targets  they  identified  (94.8  percent  and  81.8  percent,  respec¬ 
tively)  than  they  did  of  the  H9  and  L36  targets  they  identified  (15.2  per¬ 
cent  and  18.4,  percent,  respectively),  p  =  .0001.  The  interaction  between 
heuristics  and  type  of  target  indicates  that  teams  with  and  without  heuris¬ 
tics  completed  the  same  proportion  of  H21  targets  they  identified;  however, 
teams  with  heuristics  completed  a  large  proportion  of  the  L84  targets  they 
identified  and  a  smaller  proportion  of  the  H9  and  L36  targets  they  identi¬ 
fied  than  did  teams  without  heuristics  (Table  8),  p  =  .0001. 
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PROPORTION  OF  UNCERTAIN  TARGETS  QUERIED 


WITH  HEURISTICS 


SESSION 

Figure  4.  Proportion  of  Uncertain  Targets  Queried  as  a  Function  of 
Heuristics  and  Session 


TABLE  7.  PROPORTION  OF  UNCERTAIN  TARGETS  QUERIED 
AS  A  FUNCTION  OF  HEURISTICS  AND  SESSION 


Session  1  Session  2  Session  3 


Session  4 


With 

Without 


57.7  Percent 
32.1  Percent 


57.3  Percent  57.4  Percent  59.1  Percent 

32.1  Percent  37.6  Percent  41.3  Percent 


TABLE  8.  PROPORTION  OF  QUERIED  UNCERTAIN  TARGETS  COMPLETED 
AS  A  FUNCTION  OF  HEURISTICS  AND  TYPE  OF  TARGET 


H21 

H9 

L84 

L36 

With 

93.8  Percent 

0.3  Percent 

92.6  Percent 

0.4 

Percent 

Without 

95.8  Percent 

50.1  Percent 

71.1  Percent 

36.3 

Percent 
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Section  4 
DISCUSSION 


The  purpose  of  this  study  was  to  explore  the  impact  of  various  graphic  pre¬ 
sentations  and  experimenter  provided  decision  heuristics  on  team  decision 
making  in  the  context  of  a  dynamic  situation  assessment/resource  allocation 
task  which  incorporates  both  certain  and  uncertain  situations. 

EFFECTS  OF  HEURISTICS 

The  most  important  findings  of  the  study  involve  the  positive  effects  of 
encouraging  the  discussion  and  use  of  experimenter-provided  heuristics. 
Overall,  teams  with  heuristics  had  higher  scores  than  teams  without  heuris¬ 
tics.  The  advantage  of  using  the  heuristics  was  greater  on  moderate  trials 
than  on  fast  trials,  possibly  because  teams  needed  time  to  reflect  on  and 
deliberately  apply  the  heuristics.  Results  relating  to  each  of  the  four 
heuristics  provided  suggest  that  they  varied  in  usefulness. 

Teams  were  first  encouraged  to  look  for  high  value  certain  targets  and  com¬ 
binations  of  targets.  Teams  with  and  without  heuristics  completed  about 
the  same  number  of  15-point  targets,  and  teams  with  heuristics  actually 
completed  fewer  8-point  targets  than  did  teams  without  heuristics.  Subjec¬ 
tive  evidence  from  reviewing  videotapes  of  the  experimental  sessions  sug¬ 
gests,  however,  that  teams  with  heuristics  may  have  been  more  appropriately 
selective  in  completing  8-point  targets.  Taken  by  themselves,  8-point  tar¬ 
gets  are  only  of  moderate  value;  the  computer  model  completes  them  almost 
exclusively  in  combination  with  other  targets  for  the  third  team  member, 
maximizing  the  total  points  earned  by  the  team.  Combinations  of  targets 
are  more  difficult  to  identify  than  are  single  targets,  though;  and  teams 
with  heuristics  tended  to  complete  more  combinations  of  8-point  targets 
with  3-  or  5-point  targets  than  did  teams  without  heuristics.  It  should 
also  be  noted  that  teams  with  heuristics  completed  as  many  8-point  targets 
as  did  teams  without  heuristics  on  slower  trials  when  more  time  was 
available  to  look  for  combinations,  and  they  completed  as  many  as  did  teams 
without  heuristics  in  sessions  3  and  4  when  teams  without  heuristics 
started  turning  more  to  uncertain  targets. 


28 


The  clearest  effect  of  heuristics  was  associated  with  the  instruction  to 
use  expected  value  in  handling  uncertain  targets.  As  Kahneman  and  Tversky 
(1979)  suggested,  teams  operating  without  experimenter-provided  heuristics 
tended  to  select  certain  gains  over  merely  probable  gains  with  comparable 
or  even  higher  expected  value.  For  example,  teams  without  heuristics  got 
more  of  their  points  from  certain  targets  than  did  teams  with  heuristics. 
Teams  without  heuristics  also  queried  fev^er  uncertain  targets  and  were  less 
selective  in  completing  them  than  teams  with  heuristics.  Strategies  among 
the  teams  without  heuristics  for  completing  uncertain  targets  included 
variations  on  completing  all  the  targets  they  queried,  completing  the  high 
probability  targets,  completing  the  high  expected  value  targets,  completing 
only  the  H21  targets,  and  not  even  querying  uncertain  targets.  By  con¬ 
trast,  teams  with  heuristics  consistently  queried  and  completed  uncertain 
targets  at  rates  appropriate  to  a  strategy  based  on  expected  value. 

The  heuristics  also  suggested  ignoring  low-value  targets  and  staying  syn¬ 
chronized.  Teams  with  heuristics  did  complete  fewer  of  the  low-value  tar¬ 
gets.  Although  the  finding  was  statistically  significant,  the  small  number 
of  targets  involved  and  their  low  values  meant  that  the  difference  had 
little  impact  on  differences  in  the  overall  scores.  Team  synchronization 
was  not  statistically  analyzed,  but  reviews  of  the  videotapes  suggest  that 
teams  with  heuristics  lost  synchronization  less  frequently  and  for  shorter 
durations  than  those  without  heuristics.  Teams  with  heuristics  concen¬ 
trated  more  on  3-person  targets  (15-point  and  uncertain  targets),  which 
helped  to  keep  the  team  synchronized. 

OTHER  FINDINGS 

Presentation  style  did  not  significantly  influence  performance  on  the 
TRAP.  One  explanation  might  be  that  targets  remained  on  the  screen  long 
enough  for  teams  to  deliberately  identify  target  values.  Good  performance 
did  not  rely  on  instant  identification  of  targets  for  which  attention 
theory  would  be  more  applicable.  Only  one  clear  finding  relating  to  pres¬ 
entation  style  emerged;  teams  using  the  original  style  completed  more 
1-point  targets  than  teams  using  any  other  style.  The  original  style  had 
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the  most  complicated  coding  scheme,  and  teams  using  it  apparently  forgot 
the  values  of  some  targets  while  performing  the  task. 

The  three-way  interaction  among  presentation  style,  heuristics,  and  session 
in  the  analysis  of  points  gained  from  certain  targets  suggests  that  presen¬ 
tation  style  may  have  affected  the  rate  at  which  teams  with  and  without 
heuristics  moved  toward  a  common  strategy  toward  uncertain  targets.  There 
is  no  theoretically  based  explanation  for  this  pattern  of  effects.  An 
alternative  explanation  involves  the  number  of  teams  participating  and  the 
variety  of  strategies  adopted  by  teams  without  heuristics.  Of  the  teams 
without  heuristics,  only  four  teams  used  each  of  the  presentation  styles. 
Among  the  16  teams  without  heuristics,  at  least  four  distinct  strategies 
(with  multiple  variations  and  rates  of  adoption)  toward  uncertain  targets 
emerged.  It  is  possible  that  coincidental  alignments  of  similar  strategies 
among  teams  using  common  presentation  styles  could  account  for  the  reported 
i nteractions. 

Evidence  of  learning  effects  and  time  pressure  effects  emerged  as 
expected.  The  effects  of  session  on  several  variables  suggest  that 
learning  continued  beyond  the  training  session.  Specifically,  teams  seemed 
to  become  more  efficient  at  responding  to  targets  in  session  2  and  to  make 
significant  shifts  in  strategy,  especially  toward  completing  uncertain  tar¬ 
gets,  by  session  3.  As  in  previous  TRAP  studies,  time  pressure  had  a  sig¬ 
nificant  impact  on  performance.  Teams  completed  more  of  each  type  of 
target  and  scored  more  points  on  moderate  trials  than  on  fast  trials. 

The  TRAP  and  its  potential  adaptations  provide  a  rich  environment  in  which 
to  examine  the  dynamics  of  team  decision  making.  Additional  research  could 
explore  the  effects  of  varying  the  team  structure  through  assigning  differ¬ 
ent  team  member  roles  (functions),  constructing  teams  based  on  levels  of 
expertise,  applying  constraints  on  communication  patterns,  and  so  on. 
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Appendix  A 
CONSENT  FORM 


I,  _ _  _ having  full  capacity  to  consent,  do 

hereby  volunteer  to  participate  in  a  research  study  entitled,  "Team 
Resource  Allocation  Problem,"  under  the  direction  of  Ms.  Denise  L.  Wilson, 
Dr.  Clifford  Brown,  and  Major  D.  J.  McBride.  The  decision  to  participate 
in  this  research  is  completely  voluntary  on  my  part.  No  one  has  coerced  or 
intimidated  me  into  participating  in  this  program.  I  am  participating 

because  I  want  to.  _ _ _ has  adequately 

answered  any  and  all  questions  1  have  about  this  study,  my  participation, 
and  procedures  involved.  I  understand  that 

will  be  available  to  answer  any  questions  concerning  procedures  throughout 
this  study.  I  understand  that  if  significant  new  findings  develop  during 
the  course  of  this  research  which  may  relate  to  n\y  decision  to  continue 
participation,  I  will  be  informed.  I  further  understand  that  I  may  with¬ 
draw  this  consent  at  any  time  and  discontinue  further  participation  in  this 
study  without  prejudice  to  my  entitlements.  1  also  understand  that  the 
medical  monitor  of  this  study  may  terminate  my  participation  in  this  study 
if  he  or  she  feels  this  to  be  in  my  best  interest. 

I  understand  that  participation  in  this  study  may  be  photographed,  filmed 
or  videotaped.  I  consent  to  the  use  of  these  media  and  understand  that  any 
records  of  my  participation  in  this  study  may  only  be  disclosed  according 
to  federal  law,  including  the  Federal  Privacy  Act,  5  DSC  552a,  and  its 
implementing  regulations. 

I  understand  that  my  entitlement  to  medical  care  or  compensation  in  the 
event  of  injury  is  governed  by  federal  laws  and  regulations,  and  if  I 
desire  further  information  I  may  contact  _ . 

I  FULLY  UNDERSTAND  THAT  I  AM  MAKING  A  DECISION  WHETHER  OR  NOT  TO  PARTICI¬ 
PATE.  MY  SIGNATURE  INDICATES  THAT  I  HAVE  DECIDED  TO  PARTICIPATE  HAVING 
READ  THE  INFORMATION  PROVIDED  ABOVE. 


A.M. 

_  P.M. 

Signature  Date  Time 

I  have  briefed  the  volunteer  and  answered  questions  concerning  the  research 
project. 


Signature 


Date 
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ADDENDJ'l  TO  THE  CONSENT  FORM 


Experiment:  Team  Resource  Allocation  Problem 

You  are  invited  to  participate  in  an  experiment  designed  to  study  how 
people  in  teams  work  with  one  another  to  complete  a  number  of  tasks.  The 
situation  you  will  be  exposed  to  has  theoretical  similarity  to  those 
encountered  in  command,  control,  and  communications  (C^)  systems  of  the 
U.S.  Air  Force.  A  better  understanding  of  the  processes  by  which  team  mem¬ 
bers  complete  tasks  will  assist  in  improving  these  systems.  Your  expo¬ 
sure  to  the  equipment  is  limited  to  your  watching  the  CRT  screen  at  a  dis¬ 
tance  of  about  two  feet  for  approximately  two  hours  per  day  for  three 
days.  This  does  not  involve  any  known  risks. 

In  the  experiment,  you  will  be  observing  a  computer  generated  display  of  a 
representation  of  a  work  environment.  By  pressing  pushbuttons  on  a 
response  box,  you  will  work  on  tasks  individually  and  with  your  team  mem¬ 
bers.  Because  there  will  be  more  tasks  available  to  you  than  you  can  com¬ 
plete,  the  particular  tasks  you  and  your  team  members  choose,  and  when  you 
choose  them,  will  be  of  primary  interest.  You  will  receive  further 
detailed  instructions  at  the  beginning  of  the  experiment. 

The  responses  you  make,  and  the  times  at  which  you  make  them  will  be 
recorded  for  later  analysis.  Audio  and  video  recordings  will  also  be  made 
for  subsequent  study.  Your  name  will  be  recorded  along  with  the  dates  and 
times  at  which  the  experiment  is  performed.  Your  confidentiality  as  a  par¬ 
ticipant  in  this  project  will  be  protected.  Your  identity  will  only  be 
revealed  in  accordance  with  the  Privacy  Act,  5  DSC  552,  and  its  imple¬ 
menting  regulations.  A  numeric  code  will  be  used  to  identify  the  data  in 
any  publication. 

Any  monetary  benefits  will  be  in  accordance  with  SRL/Ai r  Force  agreements. 

You  are  free  to  refuse  to  participate  or  to  withdraw  your  participation  in 
the  experiment  at  any  time.  Doing  so  will  not  prejudice  your  relation  with 
the  laboratory  in  any  respect. 

Any  questions  you  may  have  should  be  directed  to  Ms.  Denise  Wilson  (57572) 
or  Major  D.  J.  McBride  (57570). 

You  willingness  to  participate  in  this  experiment  is  greatly  appreciated. 
Your  signature  indicates  that  you  have  decided  to  participate,  having  read 
the  information  provided  above. 

YOU  WILL  BE  GIVEN  A  COPY  OF  THIS  FORM  TO  KEEP. 


Si gnature  Date 
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Appendix  B 

SAMPLE  TRAP  INSTRUCTIONS 


ALPHABETIC  CODING 

Before  you  take  seats,  please  take  a  card  which  will  determine  whether  you 
will  be  team  member  A,  B,  or  C  for  the  study.  (Subjects  draw  cards  and  are 
seated  at  appropriate  seats.)  Each  team  member  has  his/her  own  display  and 
control  box  at  the  workstation.  You  will  communicate  with  your  teammates 
through  the  headphones  proved  at  the  workstation.  The  microphones  on  the 
headsets  are  called  lip  mikes  because  they  are  intended  to  be  kept  nearly 
touching  your  lips.  To  hold  the  background  noise  to  a  minimum  and  to  hear 
your  voices  clearly,  please  remember  to  keep  the  mike  in  this  position. 
(Demonstrate) 

GENERAL  INSTRUCTIONS 

This  experiment  is  concerned  with  how  team  members  work  with  one  another  to 
accomplish  tasks.  You  and  your  teammates  will  work  together  to  decide  how 
best  to  allocate  team  resources  (your  work  time)  for  the  good  of  the  team 
in  a  task  which  involves  the  processing  of  various  targets.  You  will  work 
on  some  targets  yourself,  and  on  other  targets  with  one  or  both  of  your 
teammates.  The  major  portion  of  the  display  will  have  11  rows.  Targets, 
represented  as  circles  labelled  A,  B,  C,  D,  and  E  will  appear  randomly  in 
each  of  these  11  rows.  You  will  earn  points  for  the  team  by  working  on 
these  targets  before  their  time  runs  out  and  they  leave  the  screen. 

Working  on  a  target  simply  means  selecting  a  target  by  using  the  buttons  on 
your  control  box  to  move  a  cursor,  pressing  the  start  button,  and  waiting  a 
few  seconds  for  the  target  to  be  processed.  Because  more  targets  than  you 
can  possibly  work  on  will  appear  on  the  screen,  the  particular  targets  you 
choose,  and  their  point  values  to  the  team,  will  be  quite  important. 
Therefore,  it  is  necessary  for  you  to  learn  how  the  point  values  of  targets 
are  determined.  Please  listen  carefully. 
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ALPHABETIC  CODING 


The  point  value  of  each  target  depends  on  two  things;  the  number  of 
required  workers  and  the  letter  that  appears  on  the  target.  Overall,  the 
point  value  of  a  target  is  proportional  to  the  number  of  required 
workers.  The  average  value  of  all  the  targets  is  3  points  per  person. 
Targets  requiring  one,  two,  or  three  workers  are,  therefore,  worth  an 
average  value  of  3,  6,  or  9  points,  respectively,  for  the  team.  Whether  a 
particular  target  is  worth  this  average  value  of  3  points  per  person,  or 
somewhat  more  or  less,  depends  on  the  letter  that  appears  on  the  target. 

Since  the  letter  A  suggests  importance  or  urgency,  you  will  see  that  tar¬ 
gets  coded  toward  the  begining  of  the  alphabet  are  worth  more  points  on  the 
average  than  targets  coded  later  in  the  alphabet.  With  this  notion  of 
alphabetic  order  in  mind,  let's  examine  the  specific  point  values  assigned 
to  the  different  targets.  (Give  subjects  the  point  values  table.) 

The  point  values  assigned  to  the  different  targets  are  shown  in  the  table. 

•  A  targets  are  worth  5  points  per  person. 

•  B  targets  are  worth  4  points  per  person. 

0  C  targets  are  worth  3  points  per  person. 

0  D  targets  are  worth  2  points  per  person. 

0  E  targets  are  worth  1  point  per  person. 

You  should  now  be  able  to  determine  the  point  value  of  each  target  by 
knowing  the  number  of  required  workers  and  the  letter  that  appears  on  the 
target. 

POINT  VALUE  TESTING 

Because  your  understanding  of  the  point  values  is  critical  to  this  study,  I 
am  going  to  have  you  complete  a  short  test  to  demonstrate  your  knowledge  of 
the  point  values  of  each  target.  Before  taking  the  test,  please  examine 
the  summary  table  of  the  point  values  and  feel  free  to  ask  questions  about 
it.  (Pause)  Do  you  have  any  questions  before  you  take  the  test? 


(Give  subjects  the  test.  If  any  questions  are  missed,  discuss  the  question 
with  the  subject  to  ensure  his  understanding  and  then  give  him  a  new 
test.  Repeat  this  procedure  until  all  subjects  have  answered  all  the  ques¬ 
tions  correctly.) 

We  are  ready  to  continue.  (START  APPROPRIATE  DEMO) 

DEMONSTRATION 

This  is  a  demonstration  of  the  TRAP  task.  As  you  can  see,  there  are  11 
rows  on  which  targets  appear  at  random  and  move  across  the  screen  from  left 
to  right.  The  black  squares  in  columns  A,  B,  and  C  indicate  which 
operators  are  required  to  work  on  each  target.  The  scale  at  the  top 
represents  30  time  units. 

Working  on  a  target  is  very  simple.  All  you  do  is  move  your  marker,  a 
green  asterisk,  to  a  target  row  and  press  the  start  button  on  your  response 
box.  Work  automatically  begins,  and  after  a  short  time  (10  time  units), 
your  team  will  receive  the  appropriate  number  of  points  for  completing  the 
target.  These  points  are  automatically  added  to  the  accumulated  points 
display  (show).  When  you  begin  work  on  a  target,  a  black  bar  will  appear 
in  that  target's  row.  The  bar  represents  the  10  time  units  required  to 
complete  processing  of  the  target.  The  target  will  move  through  the  bar  as 
the  processing  proceeds.  When  the  target  moves  out  of  the  bar,  processing 
of  that  target  has  been  completed.  In  order  to  complete  a  target  before  it 
leaves  the  screen,  you  will  have  to  start  it  before  it  reaches  the  black 
dashed  line  (while  the  target  is  in  the  opportunity  window). 

Work  on  each  target  can  be  done  only  by  a  particular  team  member  or  com¬ 
bination  of  team  members.  As  the  control  box  before  you  indicates,  you  are 
either  team  member  A,  B,  or  C.  You  can  work  only  on  those  targets  which 
have  a  black  square  in  your  column.  If  a  target  has  more  than  one  black 
square  in  front  of  it,  both  or  all  three  corresponding  team  members  will 
have  to  work  on  the  target  at  the  same  time  in  order  to  complete  it. 
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To  work  on  a  target,  you  must  move  your  marker  to  the  corresponding  black 
square.  You  move  the  marker  by  pressing  the  buttons  labeled  up  and  down  on 
your  control  box.  Go  ahead  and  move  your  marker  around.  Notice  that  if 
you  press  the  up  button  when  you  are  on  the  top  row,  your  marker  moves  to 
the  bottom  row.  Similarly,  if  you  press  the  down  button  when  you  are  on 
the  bottom  row,  the  marker  moves  to  the  top  row. 

Once  you  have  the  marker  on  the  row  corresponding  to  the  target  you  wish  to 
work  on,  all  you  have  to  do  is  press  the  start  button.  If  you  are  the  only 
team  member  required  for  that  target,  work  automatically  begins  and  the 
black  square  will  turn  yellow.  However,  if  one  or  more  additional  team 
members  are  required  for  the  target,  the  black  square  will  turn  pink.  This 
means  that  you  are  waiting  to  work.  Work  will  begin  only  when  all  the 
required  workers  for  the  target  have  moved  their  markers  to  the  target  and 
pressed  their  start  buttons.  When  this  occurs,  all  the  squares  will  turn 
yellow  indicating  that  work  has  begun. 

While  you  are  working  on  a  target  or  waiting  for  another  team  member  at  a 
target,  your  marker  will  turn  red.  You  can  move  it  to  any  row  you  choose 
in  preparation  for  the  next  target  you  may  wish  to  start.  When  you  become 
free,  your  marker  will  return  to  its  green  color  indicating  that  you  are 
ready  to  press  the  start  button  for  another  target. 

You  may  wish  to  stop  working  on  a  target  before  completing  it.  To  do  this, 
you  simply  press  the  RESET  button  on  your  control  box.  Your  marker  will 
turn  green  indicating  that  you  are  free  to  start  another  target.  If  others 
were  working  on  the  target  with  you,  they  will  also  have  to  press  their 
RESET  buttons  to  work  on  a  different  target.  You  will  receive  no  points 
for  targets  which  are  not  fully  completed.  If  you  choose,  you  may  begin  to 
process  the  target  over  again,  but  it  will  take  a  full  10  time  units  to 
complete  it.  The  RESET  button  is  also  used  when  you  no  longer  wish  to  wait 
for  other  team  members  at  a  particular  target. 

Processing  of  each  target  takes  10  time  units  (TUs).  A  TU  is  some  arbi¬ 
trary  number  of  seconds.  The  current  example  trial  has  a  TU  of  3  sec¬ 
onds.  It  takes  30  TUs  (in  this  example,  90  seconds)  for  a  target  to  move 
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completely  across  the  screen.  During  the  actual  experiment,  the  number  of 
seconds  for  a  TU  will  be  less.  That  is,  the  targets  will  move  across  the 
screen  more  quickly  and  the  time  spent  processing  each  target  will  be  less. 

The  table  in  the  lower  right  hand  portion  of  the  display  indicates  whether 
each  team  member  is  free,  waiting,  or  working.  A  black  square  indicates 
that  a  particular  team  member  is  free,  while  a  blinking  pink  square  indi¬ 
cates  that  a  particular  Learn  member  is  waiting.  When  a  particular  team 
member  is  working  a  numeric  countdown,  in  TUs,  will  indicate  how  much  pro¬ 
cessing  time  remains  until  the  team  member  will  be  finished  with  the  cur¬ 
rent  target. 

The  countdown  for  each  target  will  start  at  30  TUs  when  a  target  is  at  the 
left -most  part  of  the  screen,  and  decrease  at  a  constant  rate  as  the  target 
moves  to  the  right.  When  a  target  is  at  the  end  of  the  opportunity  window 
(the  black  dashed  line),  the  countdown  will  be  at  10  TUs.  The  target 
leaves  the  screen  at  0  TU.  Since  each  and  every  target  requires  10  TUs  for 
processing,  knowing  how  many  TUs  a  target  will  remain  on  the  screen  can  be 
useful  to  you  as  you  decide  which  targets  to  work  on,  and  when  to  work  on 
them.  In  addition,  comparing  this  information  to  the  countdown  of  team 
members  who  are  currently  working  (show)  can  provide  vital  information 
about  whether  there  will  be  enough  time  to  process  particular  targets.  For 
example,  if  a  team  member  has  6  TUs  remaining  before  completing  a  particu¬ 
lar  target,  he  will  not  be  able  to  complete  both  that  target  and  another 
target  that  currently  has  only  15  TUs  remaining. 

The  object  of  this  exercise  is  to  accumulate  as  many  points  as  possible  as 
a  team.  This  means  discussing  alternatives  with  the  other  members  of  the 
team  in  order  to  make  optimum  selection  of  targets.  As  there  will  be  more 
targets  than  the  team  can  possibly  process,  combinations  of  targets  should 
be  selected  which  optimize  team  performance  and  total  point  count. 
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TRAP  Task  Point  Values 


Target  (T)  Q)  (J)  Q)  Q) 

Point  Value  123  45 

(Per  Person) 
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Appendix  C 

INSTRUCTIONS  FOR  TRAP  WITH  UNCERTAIN  TARGETS 

Now  that  you  are  familiar  with  the  basic  TRAP,  we  would  like  to  introduce  a 
variation.  In  order  to  study  how  teams  perform  with  incomplete  or  uncer¬ 
tain  information,  we  have  added  targets  for  which  the  point  value  is  ini¬ 
tially  unknown.  The  team  may  choose  to  query  the  system  to  gain  more 
information  about  these  targets  before  deciding  whether  to  commit  to  pro¬ 
cessing  them. 

The  UNCERTAIN  TARGETS,  which  appear  on  the  screen  as  black  rectangles,  all 
require  three  persons  to  process  them.  There  are  two  features  of  these 
targets  which  may  vary; 

1.  Probability  of  Payoff;  The  probability  that  a  team  will 
actually  get  the  points  for  processing  the  uncertain  target  is 
either  80  percent  (high)  or  20  percent  (low). 

2.  Point  Value;  There  are  two  possible  point  values  for  each  level 
of  probability  of  payoff. 

These  two  features  taken  together  result  in  four  types  of  uncertain  types 
as  shown  as  follows; 

Probability  of  Payoff  Point  Values 

High  (80  Percent)  7  (21)  3  (9) 

Low  (20  Percent)  28  (84)  12  (36) 

Targets  with  a  high  probability  of  payoff,  then,  have  an  80  percent  proba¬ 
bility  of  giving  points  and  may  be  worth  either  7  points  per  person 
(21  points  for  the  team)  or  3  points  per  person  (9  points  for  the  team). 
Targets  with  a  low  probability  of  payoff  have  a  20  percent  probability  of 
giving  points  and  may  be  worth  either  28  points  per  person  (84  points  for 
the  team)  or  12  points  per  person  (36  points  for  the  team).  Each  of  the 
four  types  of  uncertain  targets  is  equally  likely  to  occur. 


39 


There  is  a  cost  in  time  (2  TUs)  required  to  get  information  about  the  tar¬ 
gets.  Taking  into  account  the  time  cost,  the  uncertain  targets  are  on  the 
average  about  equivalent  to  a  10-point  three-person  target  (3.3  points  per 
person) . 

Initially,  the  uncertain  targets  appear  on  the  screen  as  black  rectangles 
with  no  information  about  the  probability  of  payoff  or  point  value.  In 
order  to  query  for  information,  all  three  members  must  move  their  markers 
to  that  row  and  press  their  start  buttons,  just  as  you  would  to  work  on  a 
target.  Two  TUs  after  the  three  team  members  have  pressed  their  start  but¬ 
tons,  information  about  the  target  will  appear  on  the  black  rectangle.  The 
information  will  include  either  an  H  or  L,  for  high  or  low  probability  of 
payoff,  respectively,  and  the  total  number  of  points  the  team  may  earn  for 
processing  the  target. 

At  the  time  the  information  is  obtained,  the  team  may  choose  to  process 
that  target,  which  they  can  initiate  by  pressing  their  start  buttons  a 
second  time  (uncertain  targets,  like  the  other  TRAP  targets,  require  10  TUs 
for  processing),  or  they  may  choose  not  to  process  that  target,  in  which 
case  they  are  free  to  move  their  markers  to  another  target  or  targets.  The 
team  may  choose,  if  time  allows,  to  query  for  information  on  more  than  one 
uncertain  target  before  selecting  a  target  or  targets  to  process. 

The  rules  and  procedures  for  processing  the  uncertain  targets  are  exactly 
the  same  as  those  for  the  standard  TRAP  targets. 

As  with  the  basic  TRAP,  the  object  of  the  task  is  to  accumulate  as  many 
points  as  possible  as  a  team.  This  means  discussing  alternatives  with  the 
other  members  of  the  team  in  order  to  make  optimum  selection  of  targets. 

The  uncertain  targets  may  be  viewed  simply  as  additional  alternatives.  As 
there  will  be  more  targets  than  the  team  can  possibly  process,  combinations 
of  targets  should  be  selected  which  optimize  team  performance  (total  point 
count). 
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Appendix  D 

INSTRUCTIONS  FOR  TRAP  WITH  HEURISTICS 


A  heuristic  is  a  "rule  of  thumb"  or  a  simplified  approach  to  a  problem  that 
grows  out  of  experience  with  similar  situations.  It  may  not  always  lead  to 
the  best  solution  to  a  given  problem,  but  it  generally  provides  a  good 
solution  that  can  be  identified  quickly  and  easily.  For  example,  an  appro¬ 
priate  heuristic  for  the  TRAP  task  might  be  "Always  work  3-person  A  tar¬ 
gets."  People  who  have  had  experience  with  TRAP  have  suggested  several 
heuristics  for  TRAP,  and  we  encourage  you  to  discuss  and  to  use  the  heuris¬ 
tics  as  you  start  to  perform  the  task. 

HEURISTIC  #1:  In  addition  to  a  3-person  A  targets,  look  for  a  2-person 
B  target,  especially  with  a  1-person  A  target. 

A  2-person  B  target  worked  at  the  same  time  as  a  1-person  A  target 
gets  13  points  for  the  team.  Of  the  "for  sure"  targets,  only  a 
3-person  A  target  or  three  1-person  A  targets  worked  at  the  same  time 
get  more  points.  Even  a  1-person  C  target  worked  at  the  same  time  as 
a  2-person  B  target  gets  more  points  (11)  than  many  combinations  that 
may  be  available. 

HEURISTIC  #2:  Next,  check  as  many  uncertain  targets  as  you  can.  Immedi¬ 
ately  take  L84  or  H21;  ignore  L36  and  H9. 

Heuristic  #2  comes  from  the  expected  value  of  the  uncertain  targets, 
or  the  average  payoff  you'll  get  for  working  those  targets  over 
time.  Taking  into  account  the  time  delay  to  identify  the  targets,  the 
uncertain  targets  are  worth  on  average  about  10  points  for  the  team. 
(In  other  words,  you  should  take  combinations  of  "for  sure"  targets 
that  are  worth  more  than  10  points  before  you  check  uncertain  tar¬ 
gets.)  You  can  compute  the  expected  value  of  an  individual  uncertain 
target  by  multiplying  the  probability  of  payoff  by  the  points  availa¬ 
ble.  For  example,  the  expected  value  of  an  L84,  once  you  identify  it, 
is  (0.2)  x  84  =  16.8.  Therefore,  an  L84  will  only  pay  off  one  out  of 
five  times,  but  the  payoff  is  big  when  it  comes.  Your  score  on  one 
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trial  may  not  show  the  benefit  of  selecting  an  L84,  but  your  overall 
average  score  will.  An  H21  also  gets  an  expected  value  of  16.8,  so  it 
is  just  as  valuable  as  an  L84.  The  expected  value  of  an  L36  or  an  H9, 
though,  is  only  7.2,  which  is  so  low  that  there  are  almost  always 
better  options  available. 

HEURISTIC  #3:  Ignore  D  and  E  targets. 

There  will  almost  always  be  a  better  combination  available  than  one 
that  involves  1-  and  2-point-per-person  targets. 

HEURISTIC  #4:  Keep  the  team  synchronized.  All  three  team  members  should 
start  a  target  or  targets  at  the  same  time. 

If  you  can't  start  working  a  target  very  close  to  the  same  time  as 
your  teammates,  you're  better  off  to  stay  free  until  they  are  free 
again  than  to  start  working  a  1-person  target  that  comes  along  after 
they've  started.  You'd  just  end  up  in  an  endless  round  of  waiting  for 
someone  to  be  free  to  start  team  targets. 

Do  you  have  any  questions  about  the  heuristics?  (Answer  questions)  You 
may  now  have  5  minutes  to  discuss  the  heuristics  and  any  other  strategies 
you  might  choose  to  adopt.  (Give  subjects  the  summary  of  heuristics.) 

HEURISTIC  #1:  In  addition  to  3-person  A  targets,  look  for  a  2-person 
B  target,  especially  with  a  1-person  A  target. 

HEURISTIC  #2:  Next,  check  as  many  uncertain  targets  as  you  can.  Immedi¬ 
ately  take  L84  or  H21;  ignore  L36  and  H9. 

HEURISTIC  #3:  Ignore  D  and  E  targets. 

HEURISTIC  #4:  Keep  the  tcum  synchronized.  All  three  team  members  should 
start  a  target  or  targets  at  the  same  time. 
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Appendix  E 

TEAM  RESOURCE  ALLOCATION  PROBLEM 
TEST  OF  POINT  VALUE  UNDERSTANDING 


VERSION  3 


Name _ Team _ Worker:  ABC 

Date  Session 


One 

-Person  Events 

1. 

If  A  processes 

© 

the 

team 

win 

accumulate  _ 

_ poi nts . 

2. 

If  B  processes 

© 

the 

team 

will 

accumulate  _ 

_  points. 

3. 

If  C  processes 

® 

the 

team 

will 

accumulate  _ 

_  points. 

Two 

-Person  Events 

4. 

If  AB  together 

process 

the 

team 

win 

accumulate  _ 

_  points. 

5. 

If  BC  together 

process 

the 

team 

wi  n 

accumulate  _ 

_  points. 

Three-Person  Events 

6. 

If  ABC  together 

process 

the 

team 

wi  1 1 

accumulate  _ 

_  points. 

7. 

If  ABC  together 

process 

the 

team 

will 

accumulate  _ 

_  points. 

8. 

If  ABC  together 

process 

the 

team 

will 

accumulate  _ 

_  poi nts. 
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Appendix  F 

lEm  RESOURCE  ALLOCATION  PROBLEM 
TEST  OF  HEURISTICS  UNDERSTANDING 


VERSION  3 


Name _ Team _ Worker:  ABC 

Date  Session 


Each  of  the  following  diagrams  represents  a  limited  version  of  a  situation 
you  could  face  in  the  TRAP.  Using  the  heuristics  you  have  learned,  examine 
each  diagram,  select  the  action  that  is  most  appropriate,  and  respond  to 
the  question  following  the  diagram. 


Please  indicate  which  line  or  combination  of  lines  your  team  should  work 
on: 


Please  indicate  which  line  or  combination  of  lines  your  team  should  work 
on: 
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5 


ABC 


■ 

□ 

■ 

©  [ 

□ 

■ 

■ 

©  1 

■ 

□ 

□ 

®  1 

■ 

■ 

n 

©  1 

□ 

■ 

□ 

®  1 

□ 

□ 

□ 

■  H  9k  1 

Please  indicate  which  line  or  combination  of  lines  your  team  should  work 
on: 


ABC 


1  H 

■ 

© 

■ 

T.  1  1 

■ 

■■i 

4 

© 

=  □ 

P 

1 

m 

Please  indicate  the  best  action  for  person  C  (place  an  X  next  to  the  best 
option) : 


Get  team  to  reset  and  start  work  on  line  1. 

Suggest  line  2  as  next  target  after  completing  line  5. 
Suggest  line  3  as  next  target  after  completing  line  5. 
Start  work  on  line  4. 

Get  team  to  reset  and  start  work  on  line  6. 
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